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Council's (DETC's) accreditation process, details the process of writing a 
self-evaluation report (SER) . In the introduction, the SER is defined as the 
heart of the DETC accreditation process and is described as a road map for 
institutions in developing truly self -analytical and honestly introspective 
assessments of their present and future. Next, benefits of writing an SER are 
listed along with suggestions for developing an analytical writing style when 
completing an SER. A sample nonanalytical response to an evaluation question 
is provided and compared with an analytical response to the same question. 

The issue of selecting areas to analyze in an SER is examined, and the 
differences between internal and external analyses are illustrated through 
several examples. Sixteen topics are listed as possible topics for inclusion 
in an analytical SER. Among the topics listed are the following: existence of 
employment opportunities for the institution's graduates; appropriateness of 
instructional materials' reading level; validity/reliability of tests; 
usefulness of remarks on test papers; possible weaknesses in student 
services/administration that are causing dissatisfaction among students; and 
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Introduction 

It’s been called the heart of the accreditation process ... a useful 
planning document ... a valuable training tool for new staff ... a 
repository for an institution’s corporate memory ... a public rela- 
tions resource . . . management’s window on the inner workings of 
the business ... a reason for conducting a formal internal evaluation 
of all activities in a school ... a rationale and blueprint for institu- 
tional renewal . . . and a road map for future survival. 

An SER is all these things and much more! 

The Self Evaluation Report (SER), which the Accrediting Commis- 
sion of the Distance Education and Training Council requires each 
applicant for accreditation to submit every five years, has been rela- 
tively unchanged in the past 40 years. 

However, with the publication in the Spring 1995 of the DETC Ac- 
creditation Handbook, the “Guide to Self-Evaluation” document has 
undergone a significant overhaul. It has been expanded in scope and 
depth of coverage. It not only asks several new questions, it also 
challenges applicants for accreditation to present truly self-analyti- 
cal responses to a number of important questions. 

The preamble to the 1995 “Guide to Self-Evaluation” states: 

The SER serves as a document that tells a story about your institu- 
tion, including how the institution originated, what your institution 
offers the field of distance education and training, how it is managed, 
how it complies with accreditation standards and requirements, and 
what the institution’s plans are for future improvement and growth. 

It should also present in a business-like way insigh ful analyses into 
trends and patterns within an institution and discuss candidly the 
larger issues and challenges confronting it. An SER is, fundamentally, 
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a road map for institutions in developing a truly self-analytical and 
honestly introspective picture of how they are and where they are go- 
ing. 

The “Guide to Self-Evaluation” document in the DETC Accredita- 
tion Handbook provides specific directions on how to prepare an 
SER in terms of format and physical appearance. But what may not 
be so clearly disclosed is the subject addressed in this Occasional 
Paper. How do you prepare an SER that will be the best possible 
document for accomplishing the critical task it is supposed to 
achieve, namely, ensuring the institution’ s accreditation or re- 
accreditation? 

Outside observers of the DETC accreditation procedure for self- 
evaluation have high praise for the vast amounts of data and informa- 
tion which the Guide solicits. However, one weakness noted in many 
Self-Evaluation Reports was that they “were not very analytical, or 
introspective or evaluative.” 

A good SER presents more than mere information and data. It goes 
well beyond the “bare bones,” businesslike style of responding to 
questions with brevity. 

A good SER offers the intended reader — the visiting on-site evalua- 
tor — data and interpretation, information and insight and a fact-based 
presentation of the institution’s strengths and weaknesses. It also of- 
fers the reader a sense of what the institution is all about. 

Some reports are long on words but short on insight. They seem to 
lack genuine candor in discussing, in an open and honest way, the 
institution’s weaknesses and failings. Real problem issues are glossed 
over, and the big challenges confronting the institution are scarcely 
mentioned. 
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Yet another shortcoming in some SERs has been the failure to de- 
velop patterns and themes within the overall Report that give an ac- 
curate portrait of just what the institution stands for, where it is 
coming from and where it is heading. 

It’s as if after reading dozens of pages, the reader comes away from 
the Report with no more understanding of the institution than if he or 
she could understand the problems of Amtrak after reading train 
schedules. 

Another soft spot in many SERs is their failure to track and analyze 
patterns of issues confronting the institution over a period of years, 
and the Report’s failure to address candidly the tough questions con- 
fronting institution. Some SERs read as if they had been thrown to- 
gether at the last moment. 

And finally, too often the job of writing an SER is delegated to a 
single, lower level staff member with minimal guidance and interest 
by top management, other than to scan the Report before the visit 
by the Examining Committee. Such ill-prepared Reports can have fa- 
tal results to the institution’ s application for accreditation. Most poor 
SERs can be traced directly to top management’ s lack of apprecia- 
tion for the importance of preparing a genuinely introspective and 
analytical document, one that tells the whole story. 

Some reasons for writing analytical reports are: 

1 . A truly analytical Report will “disarm” the visiting evaluators and 
show them how the institution is solving its problems on its own. 
(The evaluators will likely discover the problems anyway ! And, 
they will likely offer their own solutions !) 

2. An analytical SER will be much more useful in — 
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• long-range strategic planning 

• genuine self-evaluation 

• internal management problem solving 

• curing chronic system problems 

• establishing open internal communications 

• creating an atmosphere of creativity among staff by involving 
them in solutions to identified problems 

• providing a platform for new product development, enhancing 
current services and stimulating continuous organization re- 
newal 

• saving precious time and efforts by describing problems and 
solutions “up front,” so that visiting examiners can focus on 
helping the institution. 

3. An analytical SER will help ensure that the Chairman’s Report 
(following the on-site accrediting visit) will include not merely 
statements of “problems” but the institution’s “solutions,” thus 
contributing to a balanced picture that includes — and is positively 
flavored with — the institution’ s perspective. 

Developing an Analytical Writing Style 

The 1995 “Guide to Self-Evaluation” presents numerous opportunities 
for SER writers to be candid and creative in presenting analytical re- 
sponses. 

Revised questions in the 1995 Guide which solicit analytical reactions 
include these: 
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• As candidly as possible, describe the significant challenges cur- 
rently facing the institution . State how the process of DETC 
self-evaluation has helped clarify /identify any major issues or 

problems. 

• Describe major institutional changes and improvements made 
since the last accreditation examination of the institution. 

• State what problems, actions, or policies should be emphasized 
to the visiting evaluators as examples of significant institution 
achievements in quality distance education and training. 

• Describe any trends and/or patterns within the institution 
which demonstrate the overall quality of the institution. 

The above questions are pointed invitations to SER writers to “open 
up” to the reader and get down to business in discussing the real is- 
sues, the life and death issues, facing the institution. These questions 
provide a platform to discuss where the institution is heading and 
how it plans to get there — without necessarily compromising the pro- 
prietary or confidential business plans of the organization. 

Here are “before” and “after” responses to SER question XII. E. 3, 
which reads: Describe the major institution changes and improve- 
ments made since the last accreditation examination of the insti- 
tution. 

A Non-Analytical Response 

In the five years since the last visit of the Accrediting Commis- 
sion, the XYZ Academy has consistently improved its courses and 
its services. It has added six new courses of study, acquired a 
new computer system and added 12 new staff. Enrollments have 
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climbed 40%. There have been no changes in top management 
or in direction, etc. . . . 

The response above is factual. But does it really offer the kind of 
response that reveals the introspection and self analysis that a good 
SER should have? Facts alone are sometimes not enough. Let’s look 
at a revised version of a response to the same question. 

An Analytical Response 

The XYZ Academy embarked in 1991 on a major campaign of 
institutional self analysis and continuous renewal. The 1990 
Chairman’s Report identified three significant areas for XYZ to 
address: updating of courses, enhancing and modernizing student 
services, and curing chronically low course completion rates. 

In addition to these three areas, XYZ had identified, in early 
1991, an additional problem that needed to be addressed: falling 
enrollment levels had been creating a cash flow crunch that was 
stifling all other efforts to upgrade the Academy. 

In order of priority, the Academy attacked the four problems in 
a systematic way. Below is a discussion of the statement of each 
problem since 1990, the central cause of the problem, the 
method of attacking the problem and the results achieved to 
date, etc. . . . 

But why go to the effort of “telling all” by discussing the problems 
facing the institution? Why not let the Examining Committee find out 
for themselves what the problems are? 

Because the Committee will find the problems anyway and will likely 
recommend their own solutions. And the solutions may not be the 
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ones that the institution is prepared to implement. It is far better to 
anticipate what problem areas are most critical and then present a 
solution in the SER. Why let a visiting group suggest solutions to 
problems that you already know — or should know? 

Institutions cheat themselves when they prepare less than candid re- 
ports. The greatest benefit of voluntarily undertaken self-evaluation is 
the formal opportunity the exercise presents to allow you to (1) de- 
termine what needs to be done, (2) describe how to accomplish it, 
and (3) show a track record of solid achievement. 

Areas to “Analyze” in Your SER 

Self-analysis can be fun. Whether you do it only for internal planning 
(nothing compels you to share your results), or to satisfy an SER re- 
quirement, do it. Whether you do it to know what changes to make 
or to satisfy yourself that no changes are needed, do it. Each institu- 
tion should consider including some intensive self-analysis of its op- 
erations on a continuing basis, not just for SER purposes. 

Your analysis may be a sophisticated statistical one with built in con- 
trols and allowable margins for error. But more probably, it will be 
the development of simple questions asked about specific problems 
confronting the institution. 

You must know your objectives before doing the analysis. What do 
you hope to learn; whether your students have developed proficiency 
in a particular skill; whether you need to install an 800 number for 
your students? 

The analysis may be internal or external. That is, you may analyze 
data already available in your internal records; or you may direct 
your survey to external sources such as ex-students, graduates, em- 
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ployers, and others. Although an internal analysis may be easier and 
quicker to accomplish, its results may have less credibility. But both 
can be useful. 

Your choice of areas to analyze may be tricky. While there are 
many pertinent subjects for analysis already suggested in the SER 
Guide, these may not be the areas in which you are the most vulner- 
able and therefore may be irrelevant. On the basis of introspectively 
asking yourself the questions listed in the Evaluator’ s Rating Forms in 
the DETC Accreditation Handbook, you should be able to identify 
areas of weakness and those in which you would honestly like to 
make improvement. To satisfy yourself that a problem truly exists or 
does not exist you should develop a plan for your analysis. 

One word of caution. If you are considering a change in your opera- 
tion in order to accomplish greater success, you must define suc- 
cess. What does it mean to succeed? For example, if your goal is to 
increase the percentage of graduates who get jobs and you are 
studying the relationship between course completion and getting a job 
you may have defined success as getting a job. Or if your goal is to 
assure that all textbooks are current, you will wish to define “cur- 
rent” before you classify the texts as “current” or “out of date.” 

Here are a few suggestions of areas in which you may wish to do an 
analysis that can be used in preparing an “analytical” SER. You 
should decide in advance: 

1 . The objective: what do you wish to learn from your research? 

2. The data needed: what information must you have to answer your 
questions? 
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3. The methodology you will use to gather your information. 

4. The definition of success . 

Example A of a problem suggested from Accrediting Commission 

Standard I. A., Description of Objectives: 

Internal 

1. Objective: To determine if course prepares graduates to pass li- 
censing exam. 

2. Data Needed: Need to know which questions (and how many) 
on internal exams match questions on licensing exams. 

3. Methodology: Have faculty do an item analysis of questions on 
licensing exam comparing number of matching items on your in- 
ternal exams. 

4. Definition of Success: Success = 80% of all questions on li- 
censing exam are included in internal tests given to students. 

External 

1. Objective: To determine if course prepares graduates to pass li- 
censing exam. 

2. Data Needed: Need to know the score results of your graduates 
on recent licensing exam. 

3. Methodology: Analyze scores to determine percent success ver- 
sus failure. Repeat for subsequent years to identify trends. 
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4. Definition of Success: Success = 90% of students receive pass- 
ing grade on externally administered licensing exam. 

Example B of problem suggested by Accrediting Commission Stan- 
dard VII. A., Admission Practices and Enrollment Agreements: 

Internal 

1 . Objective .To determine if admissions criteria successfully screen 
out applicants who will not finish the course. 

2. Data Needed: Need to know characteristics of admitted stu- 
dents. Need to know if given their high school background, sex, 
age, prior experience, physical limitations, they finished course? 

3. Methodology: Staff uses internal records to develop matrix of 
admission qualifications and outcomes data, comparing those who 
completed with those admitted. 

4. Definition of Success: Success = finishing course. 

External 

1 . Objective .To determine if admission criteria successfully screen 
out applicants who will get and keep a job. 

2. Data Needed: Need to know characteristics of admitted students 
who got jobs. Did those who got jobs keep them beyond initial 
period of employment? 

3. Methodology: Develop a survey to go to employers of gradu- 
ates. Learn if graduates retained jobs and compare reasons for 
failure with deficiencies in qualifications of students when admit- 
ted. 
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4. Definition of Success: Success = getting and keeping a job in a 
field related to course. 
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Other questions which may suggest themselves to you as you con- 
sider ways to improve the quality of your institution and as you are 
preparing your Self-Evaluation Report are: 

Example C suggested by Accrediting Commission Standard II. G., 
Teaching Devices: 

1 . Objectives: To determine if students would learn more easily if 
audio or video tapes were added to the current educational ma- 
terials? 



2. Data Needed: Need to know success ratios of students who 
have tapes added to other materials. 

3. Methodology: Develop a control group of students who have the 
benefit of tapes with a group of students using the current educa- 
tional materials. 



4. Definition of Success: Success = higher % of students using 
tapes finish course than those who do not have benefit of tapes. 

Example D suggested by Accrediting Commission Standard III. A., 

Examination Services: 

1 . Objective: To determine if students learn more when they were 
required to take tests with subjective type questions rather than 
the current objective true-false, multiple choice and other objec- 
tive tests. 
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2. Data Needed: Need to test students using only subjective tests to 
measure whether they more readily are given placement in the field, 
or more readily complete the course with a passing grade. 

3. Methodology: Develop a control group of students who continue 
to be tested with only objective questions as opposed to a group 
which is able to demonstrate its knowledge with subjective type 
tests. 

4. Definition of Success: Success = placement in career field or suc- 
cessful completion of course. 

Note: The above examples are merely intended to suggest the great 
number and varieties of ways that your analysis could be conducted. 
The assumptions made throughout are undoubtedly debatable. 

This is just a brief sampling of the types of analysis which you can 
make about your institution, and which can aid you in writing a truly 
analytical Self-Evaluation Report. 

The following page gives a list of other useful topics you can explore as you 
prepare an analytical SER. 
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Possible Topics: 

• Is there evidence that employment 
opportunities exist for graduates of 
my institution and the income level 
implied in my promotional literature 
is accurate? 

• Is the reading level of instructional 
material keyed to the reading com- 
petence of the average enrollee? 

• Are course texts sufficiently compre- 
hensive to permit students to pass 
internal examinations? 

• Are texts up-to-date? 

• Should I introduce a resident train- 
ing component to increase the suc- 
cess of my distance education 
program? 

• Are course examinations valid and 
reliable? Do they mislead students? 

• Could I increase the effectiveness of 
the learning experience through the 
use of telephone testing? 

• How can I improve the response 
time it takes to get test results back 
to students? 

• Do individualized remarks on test 
papers motivate students and in- 
crease their chances of success? 

o 
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• Is testing more effective if students 
have had the benefit of a pre-test? 

• What weakness in student services 
or administration is causing dissatis- 
faction among my students? 

• Do faculty in degree programs have 
an adequate number of hours of 
study in a degree program at the 
level they are teaching to meet fac- 
ulty requirements stated in DETC’s 
“Policy on Degree Programs”? 

• Have the evaluations of faculty per- 
formance improved over the last 2 or 
3 years? 

• Are sales materials accurate and 
honest in setting forth the stated ben- 
efits for taking a course? 

• Are the potential income claims 
stated in my literature true? Have 
they changed from last year? Can I 
document every claim— implied and 
overt— in my promotional materials? 

• How does the catalog description of 
my courses compare with actual 
curriculum content? 
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Conclusion 

In strategic planning, the plan itself may never be used, but the plan- 
ning process is indispensable. So too with preparing analytical Self- 
Evaluation Reports. The process of listing patterns, gathering data, 
analyzing trends and laying out solutions to identified problems is 
what the accreditation process is all about. 

An analytical SER is potentially one of the most powerful tools you 
have at your disposal to permit you to give your institution the one 
gift we cannot give ourselves: immortality. 
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